MACH1: nonuniform time-scale modification of speech

Time-compression techniques change the playback rate of speech without introducing pitch artifacts. However, when linear-compression techniques are used, human comprehension of time-compressed speech typically degrades at compression rates above two times real time [1]. These degradations are not due to the speech rate per se: Comprehension of linearly compressed speech often breaks down above 225 to 270 words per minute (wpm) [2], which is well below the rates at which long passages of natural speech are comprehensible (up to 500 wpm) [3]. Instead, the incomprehensibility of time-compressed speech is due to its unnatural timing. Mach1, described in Section 2, is an alternative to linear time compression. Mach1 compresses the components of an utterance to resemble closely the natural timing of fast speech. Section 3 describes our test of comprehension and preference levels for Mach1-compressed and linearly compressed speech. In Section 4, we draw our conclusions. 2 MACH1 TIME COMPRESSION Mach1 mimics the compression strategies that people use when they talk fast in natural settings. We used linguistic studies of natural speech [4,5] to derive these goals: • Compress pauses and silences the most • Compress stressed vowels the least • Compress unstressed vowels by an intermediate amount • Compress consonants based on the stress level of the neighboring vowels • On average, compress consonants more than vowels Also, to avoid obliterating very short segments, we need to avoid overcompressing already rapid sections of speech. Unlike previous techniques [6,7], Mach1 deliberately avoids categorical recognition (such as silence detection). Instead, as illustrated in Figure 1, it estimates continuous-valued measures of local emphasis and relative speaking rate. Together, these two sequences estimate what we call audio tension: the degree to which the local speech segments resist changes in rate. High-tension segments are less compressible than low-tension segments. Based on the audio tension, we modify the target compression rate to give local target compression rates. We use these local target rates to drive a standard, timescale modification technique (e.g., synchronized overlap-add [8]). 2.1 Local-Emphasis Measure We use the local-emphasis measure to distinguish among silence, unstressed syllables, and stressed syllables. Emphasis in speech correlates with relative loudness, pitch variations, and duration [9]. Of these, relative loudness is the most easily estimated. 2.1.1 Estimating local energy To estimate local emphasis, we first calculate the local energy. We simply use the frame energies from the spectrogram used in speaking rate estimation (see Section 2.2). 2.1.2 Normalizing by …